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Abstract 

The purpose of this paper is to discuss a study of the implementation and impacts of the Quality 
Programs Assurance System (QPAS) within Virginia’s Fairfax County Public Schools (FCPS). 
QPAS is a system of shared accountability for educational programs, with multiple levels of 
reporting. The primary goal of QPAS is to provide decision-makers with quality accountability 
information on instructional programs and services in the school system. The study examined 
the successes and challenges of QPAS after the first two-year cycle, at which point program 
managers completed formal review reports on their programs. Data were collected from 
program managers, evaluators, and high-level decision-makers. Results indicate that QPAS has 
impacted program manager attitudes, the organizational culture of the school system, and 
decision-making about programs. 
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Introduction 

In response to local and national interest in educational program accountability, Fairfax 
County Public Schools (FCPS), located in Northern Virginia, developed a shared accountability 
system, the Quality Programs Assurance System (QPAS). QPAS is a system of shared 
accountability for educational programs, with multiple levels of reporting. The responsibility for 
program accountability is shared across departments within the school system, including the 
people who manage educational programs as well as trained internal program evaluators. The 
primary goal of QPAS is to provide decision-makers with quality accountability information on 
all instructional programs and services in the school system. See Appendix A for a summary 
brochure that describes QPAS. 

QPAS was developed as part of the larger FCPS commitment to accountability, for 
students, schools, staff, and programs. Over the past several years, FCPS, a large, affluent and 
academically successful school district in Northern Virginia, has been transitioning to a client- 
centered model for education as well as making a commitment to data based decision-making at 
all levels. In keeping with the current nationwide commitment to accountability as manifested 
by the No Child Left Behind Act (2001), the Fairfax County School Board became interested in 
evaluation information for all instructional programs to help facilitate informed decision-making. 

However, due to the small number of trained evaluators in the Office of Program 
Evaluation (OPE) and the large numbers of educational programs (over 100) in FCPS, 
comprehensive evaluation of all instructional programs was not feasible. Therefore, OPE 
developed a multi-level system of shared accountability. The resulting system, QPAS, shares the 
responsibility for program accountability across FCPS departments, and represents district-level 
policy. Over the first two years, 75 programs were phased into QPAS. 
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The goal of QPAS is to provide quality accountability information on all FCPS 
instructional programs. QPAS consists of three levels of program accountability, with shared 
responsibility for data collection, analysis, and reporting between program managers and OPE 
(Table 1). The three levels, from basic to most complex, are documentation, review, and 
evaluation, and all levels must address a specific set of data elements about programs (e.g., 
purpose, goals, and objectives; groups targeted for impact; implementation; impact, and budget). 
Program managers have responsibility for documentation and review. In documentation, 
program managers maintain ongoing records of information about a program over the course of 
the school year that can be used for program improvement as well as to respond to questions 
about the status of the program (Horsch, 1996). After each two-year period, program managers 
complete formal written reviews of their programs, in which they compare documentation data 
across two years, as well as draw conclusions about the program and its future direction. As part 
of QPAS, OPE conducts program evaluations for programs selected by the Superintendent. 
Evaluations represent the most complex level of QPAS in terms of data collection, analysis, and 
reporting. 

Thus, program managers have responsibility for providing accurate information about 
their programs. One of the most critical components of QPAS is the requirement for clearly 
defined goals and objectives for all programs that can be understood by decision-makers as well 
as the public (Schilder, 1997). These goals and objectives then define the areas of impact 
measured for the program, as with results- or outcomes-based accountability systems (Horsch, 
1996; Schilder, 1997). However, QPAS represents a step beyond a pure outcomes-based 
accountability system because it was designed to provide information specifically for use in 
organizational decision-making (Horsch, 2002). QPAS incorporates the notions of an outcomes- 
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based accountability system and adds the requirements that data be provided about program 
implementation, training and staff development, etc. It is the intention of QPAS that program 
managers will develop increased capacity and comfort with monitoring and evaluating their own 
programs so that they can use the information to make program improvement decisions (King, 
2002 ). 

Systems like QPAS necessitate changes to the role definition for many program 
managers. The push toward data based decision-making requires program managers to collect, 
analyze and understand data about how their programs are functioning. In order for the system 
to be successful, program managers must take ownership, which requires that high-level 
administrators commit to the process as well and loan their positional “clout” to the system 
(King, 2002). Program managers need to be motivated to view this type of program monitoring 
as part of their daily responsibilities and as important in the accountability realm. However, 
those purposes are often viewed by program managers as oppositional in nature (King, 2002; 
Schilder, 1997). 

Evaluators in OPE coordinate QPAS for the school system and also work collaboratively 
with program managers to provide them technical assistance as they document and review their 
programs. As part of the commitment to the shared accountability process, a study of the 
implementation and initial impacts of QPAS was designed, so that modifications and 
improvements could be made. In addition, the study was designed to help OPE understand how 
QPAS was influencing the system in terms of building evaluation capacity and holding programs 
accountable. 

In an attempt to investigate how QPAS is functioning and its current impacts, this study 
addresses the following major research questions: 
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1) What are program managers’ attitudes about QPAS at the outset, during the second 
year, and after the first reporting cycle? 

2) How does QPAS impact the organizational culture of FCPS? 

3) How does QPAS impact decision-making about program improvement and program 
budgets? 

Method 

This study was designed to investigate how QPAS is functioning and its impacts on 
programs and the school system. The case study method was used as the overall research 
approach, with an embedded, single-case design (Yin, 1994), which considers multiple subunits 
of analysis as well as an analysis of the case as a whole. Multiple units of analysis included 
program managers, decision-makers and the entire school system. Both quantitative and 
qualitative analyses examined how QPAS was implemented, reactions of program managers and 
decision-makers, and impacts of QPAS on the school system. 

The researcher conducting the study is in the role of participant and observer. The 
researcher functions as a participant who works as a program evaluator in OPE and helps 
coordinate QPAS across the school system. The researcher functions as an observer, in that she 
collects data on QPAS from offices and departments throughout the school system. By using 
evidence including documentation, interview results, survey results, and other physical artifacts, 
the researcher triangulates the evidence gained through participant-observation. 

Participants in this study included the 63 program managers responsible for the 75 
programs phased into QPAS in the first two years, two high-level decision-makers in the school 
system (assistant superintendents), and two departmental liaisons, who had the major 
responsibility for overseeing QPAS and ensuring the quality of reports for programs in their 



O 

ERIC 



8 



QPAS 8 



respective departments. Finally, OPE staff that have provided training and technical assistance 
to program managers also participated in the study. 

Data Sources 

A variety of data sources were used to address the research questions, including 
documents, survey results and interview results, as well as observations and reflections of OPE 
staff. To answer the first research question about program manager attitudes, program managers 
were surveyed at three points in time (November 2000, Spring 2002, Spring 2003). To answer 
the second research question about the impact of QPAS on the organizational culture of FCPS, 
multiple data sources were used. Documents created to explain QPAS to various audiences were 
reviewed and analyzed. Feedback from participants, including program manager surveys and 
interviews and meetings with high-level decision-makers and OPE staff, also provided evidence 
of the organizational culture. Finally, the first set of formal review reports were analyzed. 

To answer the third research question about the impact of QPAS on decision-making, 
multiple data sources were used. The Spring 2003 program manager survey provided evidence 
of program manager decisions about program improvement. Interviews with the assistant 
superintendents revealed how they have used the Review reports to make program decisions. An 
analysis of the first set of review reports provides evidence of the recommendations made related 
to both budget and program improvement. 

Findings 

Program Manager Attitudes About QPAS 

To address the first research question, program managers were surveyed at three points in 
time. The purpose of these surveys was to gather information about program managers’ attitudes 
about the QPAS process, including their level of understanding, the perceived usefulness of 
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QPAS reports as well as the training they had received, and their suggestions about 
modifications to the process. 

Initial Survey. The first survey was conducted after the initial training in November 2000. 
This initial training provided an overview of QPAS for the program managers in the first phase 
of programs, including their roles and responsibilities. The survey specifically addressed the 
training and program managers’ understanding of QPAS. The majority of program managers in 
attendance, 26 out of 36, returned surveys, for a response rate of 72 percent. See Table 2 for a 
summary of responses to the first survey. Program managers reported very high levels of 
understanding of their roles (96 percent indicating full or some understanding) and 
responsibilities for documentation (100 percent indicating full or some understanding) and 
Review reports (96 percent indicating full or some understanding). Program managers also 
reported high levels of comfort with these roles and responsibilities at this initial stage (at least 
85 percent indicating very or somewhat comfortable). In addition, program managers reported 
being pleased with the clarity and usefulness of the training. At this initial stage, program 
managers’ major concerns were related to data collection and access, as well as the time required 
to complete QPAS requirements and how this influenced or interfered with their jobs. Many 
program managers indicated their beliefs that QPAS was ah additional responsibility, rather than 
part of their job as program manager. 

Second Survey. During the second year of QPAS, OPE conducted a variety of training 
sessions designed to facilitate data collection and reporting by program managers. At the end of 
the year, program managers were surveyed about their level of understanding of QPAS, the 
usefulness of QPAS, any assistance they had received, and any barriers they faced as they tried 
to complete QPAS requirements. Since many program managers had responsibility for multiple 
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programs, the survey was sent to 57 people (representing 75 programs), with 28 returning 
surveys (for a response rate of 49 percent). See Table 3 for a summary of responses to the 
second survey. Again, nearly all program managers (97 percent) indicated at least some 
understanding of QPAS at this point. However, only a slim majority (53 percent) indicated that 
QPAS had been at least somewhat useful to their program management. 

Additional questions were asked about the usefulness of specific assistance and support 
received related to QPAS. Program managers rated both the formal training and the informal 
technical assistance quite highly, with 90 percent responding that the training had been at least 
somewhat useful and 82 percent indicating that the technical assistance had been at least 
somewhat useful. High numbers (75 percent) reported receiving at least somewhat useful 
support from their supervisors. A slight majority (54 percent) reported receiving useful support 
in terms of data and assistance from other departments within the school system. 

Although program managers were positive about their level of understanding related to 
QPAS, they also indicated that they faced barriers and challenges as they tried to integrate QPAS 
into their work. Seventy-five percent indicated they lacked the time to dedicate to QPAS. In 
addition, 39 percent mentioned difficulties accessing the necessary data to address their program 
goals and objectives. Finally, 30 percent reported experiencing difficulties coordinating QPAS 
and budget information. The time barrier echoes the concern raised by program managers at the 
outset of implementation. 

Third Survey. The third survey was conducted in Spring 2003, after the first formal 
Review report cycle covering the first two years had been completed. The program managers 
whose programs were reviewed in the first cycle were surveyed (total of 30 program managers). 
Twenty program managers completed the survey, for a response rate of 67 percent. See Table 4 
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for a summary of responses to the third survey. This survey focused specifically on the Review 
report process. Program managers were asked about the usefulness of technical assistance 
received from OPE during the Review report process. The majority of program managers 
indicated that specific types of technical assistance were either somewhat or very useful, 
including individual (85 percent) and group meetings (90 percent), as well as email exchanges 
(80 percent). In addition, 85 percent of program managers agreed or strongly agreed that the 
formal Review report training was useful, 75 percent agreed or strongly agreed that they felt well 
prepared to complete the Review reports, and 80 percent agreed or strongly agreed that they had 
a good understanding of the QPAS requirements. 

OPE was interested in how program managers viewed the importance of QPAS to three 
groups: themselves, the Superintendent and Leadership Team (group comprised of assistant 
superintendents and directors of the regional clusters), and the School Board. Higher 
percentages of program managers felt that the Review reports were important to the Leadership 
Team (65 percent) and School Board (65 percent), than found the reports useful in their own 
work (45 percent). This finding is not surprising, given that the Review reports are submitted to 
the Superintendent, Leadership Team, and School Board as part of program accountability. Even 
after two years, many program managers continued to separate work for QPAS with work for 
their program management. In fact, a few program managers expressed concern about the value 
of QPAS in response to open-ended questions. 

Program managers were asked to describe challenges and barriers they were unable to 
resolve that they felt impacted the quality of their reports. Difficulties accessing data (35 
percent), confusion and fhistration about the QPAS requirements (20 percent) and a lack of time 
(15 percent) were mentioned as significant barriers. Program managers were also given the 
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opportunity to suggest modifications to QPAS. The most frequent suggestions (each mentioned 
by 1 5 percent of respondents) were to provide program managers with quality examples of 
completed reports and to reduce the repetition in the report generation and editing process that 
resulted from multiple layers of review. 

Taken together, the results from the three surveys suggest that although program 
managers understand the QPAS requirements and see some usefulness, many continue to 
struggle with the accountability aspects of QPAS and have not incorporated the accountability 
role into their understanding of program management. Analyzing and reporting data on their 
programs is still seen by many as something that is done for other people, rather than for 
formative uses. Data access and availability continue to represent challenges for program 
managers, who experience fhistrations when they cannot access the data they need to understand 
and report on their programs (King, 2002). QPAS is requiring collaboration among departments 
to facilitate data access, as well as the commitment to the evaluation of programs and program 
impacts. 

Organizational Culture 

The second research question addresses the impact of QPAS on the organizational culture 
of the school system. At a basic level, QPAS is changing and systematizing how accountability 
information is maintained and shared about instructional programs. Analyses of QPAS 
documents show that clear expectations have been established for the content and format of 
accountability reports. For the Review report, program managers were provided with a template 
to use that included guiding questions for them to answer about each of the data elements. Over 
time, the expectations have been clarified based on feedback from program managers and others. 
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However, each department has created some internal expectations for programs, expectations 
that vary across departments, which has led to confusion and occasional frustration. 

At the end of the initial Review report cycle, interviews were conducted with high-level 
decision-makers as well as with the departmental liaisons for QPAS. Almost all of the programs 
included in QPAS are located in two central departments and for the initial Review report cycle, 
the majority of programs included in QPAS were located in one department. The assistant 
superintendent for this department (ASl) was interviewed about the process. In response to a 
question about the benefits of QPAS, ASl felt that QPAS has provided benefits to herself and 
her staff, particularly in forcing people to become more data-driven and focused on developing 
clear goals and objectives for their programs. The greatest long-term benefit, she felt, was in the 
regular, cyclical reporting of data for programs. She stressed the need for a continued shift in 
organizational thinking, “QPAS needs to become a way of doing business - rather than being 
seen as something that is just done every two years for the School Board (ASl interview, 
3/7/2003).” 

The departmental liaison for this same department (DLl) indicated that one major benefit 
of QPAS was that program managers now understand that “they are in fact responsible for 
reporting data.” She added that she hoped consistent and quality reporting would reduce the 
“recreation of the wheel” in response to questions from the School Board (DLl interview, 
3/10/2003). One of her major concerns was that program managers have tended to focus on 
QPAS only when they have reports due, cautioning that, “if it remains a living, breathing thing, it 
has more potential benefits” (DLl interview, 3/10/2003). 

DLl felt that one of her primary roles related to QPAS was to communicate its 
importance within her department on a consistent basis. She agreed that ASl is dedicated to the 
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process, but that point was not communicated well enough to department staff from the 
beginning. In response to several questions, she reiterated the need for departments to emphasize 
the importance and usefulness of QPAS on an ongoing basis, so that it becomes part of peoples’ 
normal routines. She added that “institutionalizing QPAS is important - it requires a culture 
shift that hasn’t completely happened yet” (DLl interview, 3/10/2003). Her comments echo the 
responses of this department’s program managers to the survey questions, since many of them 
continue to indicate that QPAS is something they do in addition to their jobs, rather as part of 
their jobs. In order for QPAS to remain viable, it must remain feasible for all involved (Joint 
Committee, 1994; King, 2002). 

The assistant superintendent (AS2) and departmental liaison (DL2) from the second 
department were also interviewed. AS2 said that she is very thankful to have QPAS in place and 
was amazed at how much was accomplished in only the first two years. She acknowledges that 
QPAS represents a learning process for her staff and that she is seeing growth: “People are 
starting to ask the right questions and hopefully people will find it useful to them over the years.” 
She views QPAS as an “analytic tool for program improvement” (AS2 interview, 4/2/2003). 

DL2 was not assigned until just before the Reviews were due, but was involved with the 
report approval process. Her comments were very similar in spirit to those of DLl . In response 
to the question about benefits, she said that “QPAS has made people look at what they do, 
analyze what they do, and develop goals and objectives. It has forced them to look at data 
collection and determine what they need to do to meet goals” (DL2 interview, 3/25/2003). She 
anticipates that QPAS will also help in the long-term with program planning. 

DL2 also acknowledged that program managers in her department were resistant to the 
process, which she felt was likely due to QPAS not yet being part of the culture. She believed 
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that people were thinking, “Do I really have to do this?” (DL2 interview, 3/25/2003). In 
addition, one problem was that people were receiving conflicting messages about expectations, 
which led to frustrations. She felt that QPAS is requiring a culture shift for program managers, 
who often do not have a background in evaluation. However, AS2 thought that shared 
accountability should be part of the culture, given the District Superintendent’s commitment to 
data-driven decisions (AS2 interview, 4/2/2003). She did acknowledge that it was new to 
program managers and that the process would get easier over time. 

In addition to interviewing department staff, evaluators from OPE were surveyed about 
the technical assistance they provided program managers during the Review report cycle. The 
evaluators were asked about the greatest challenges for program managers. The most common 
challenges mentioned by evaluators related to difficulties program managers had connecting the 
discussion of program impacts to the stated goals and objectives. In addition, evaluators 
indicated that program managers had varied levels of understanding and comfort with the 
process, especially in terms of Review report requirements. Successful strategies evaluators used 
to help program managers generate quality reports were providing quality examples, making 
themselves available to program managers as early as possible, as well as while they were 
writing the reports. 

In general, comments from program managers as well as the departmental decision- 
makers were quite positive about the assistance provided by OPE. In order for a system such as 
QPAS to survive, it has to be made feasible for those involved in the evaluation and monitoring 
of programs. AS2 commented on the usefulness of assistance from OPE, and that it is “critical 
for the departments to not feel alone for QPAS to have ongoing success” (AS2 interview. 
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4/2/2003). Both assistant superintendents and their departmental liaisons suggest that OPE has 
provided the support to make QPAS happen. 

It appears from these meetings and interviews with people involved with QPAS at 
different levels that QPAS is begiiming to impact the organizational culture and attitudes within 
the school system in three major ways: an increasing acceptance of the concept of shared 
responsibility for program accoimtability, a gradual shift from viewing QPAS as a burden to 
viewing QPAS as a useful tool for program management and decision-making, and a shift 
toward more interdepartmental cooperation and sharing of data, with a lessening of the “passing 
the buck” phenomenon when information is not readily available. 

The interviews with departmental liaisons indicated that resistance to the changes 
necessitated by QPAS continues. While resistance to these changes still exists, acceptance at the 
higher levels will translate into acceptance, either forced or voluntary, for the program managers 
themselves. Since the Leadership Team and School Board are starting to make decisions about 
programs based on QPAS data and reports, program managers will have evidence of the 
importance of this information. However, until the program managers see the value of the 
process for their own program monitoring and management purposes, many will still view QPAS 
as an added responsibility, rather than part of their ongoing work. 

QPAS and Decision-Making 

The third research question addresses the impact of QPAS on decision-making. QPAS 
data and reports can be used to make decisions at multiple levels, from program modification and 
improvement decisions made by program managers themselves to policy and funding decisions 
made by the School Board and other high-level decision-makers. Formal updates on the QPAS 
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process were provided to the district superintendent, assistant superintendents, and School Board 
by OPE in June 2002 and March 2003. 

ASl and AS2, the assistant superintendents of the departments in which QPAS programs 
are located, were asked about the importance of QPAS. In interviews, they explained that they 
recognized the importance of using QPAS to facilitate data based decision-making. ASl 
explained that QPAS is important, but only when the data and reports are used in important ways 
to modify and improve programs. ASl then explicitly acknowledged her role in this shared 
accountability system, “My role has been to convince people that this is serious and important 
and to make decisions using the information (ASl interview, 3/10/2003).” When asked how the 
school system will keep QPAS viable, ASl immediately responded that QPAS data and reports 
must be used by staff at all levels, so that people can see the value of their efforts. 

As a decision-maker, AS2 said that she has found the Review reports extremely useful, 
for they provide her with overview and refresher information about the many programs in her 
department. AS2 acknowledged the importance of using the reports to make decisions, so that 
people see the value in their efforts. “Program managers and their supervisors need to use QPAS 
to plan and ask questions about the programs” (AS2 interview, 4/2/2003). 

DLl and DL2 were also asked about how QPAS information can inform decision- 
making. In order to make QPAS survive, DL2 stressed that ongoing and clear communication 
between all involved will be necessary, including feedback on how the reports are used in the 
decision-making process. “People need to know that what they did has been useful, to show that 
this is not a waste of time” (DL2 interview, 3/25/2003). 

Everyone interviewed stressed how important it was for staff at all levels to actually use 
the reports. DLl expressed the hope that the School Board could use the reports and in the 
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process become better informed about the programs. Having an informed School Board would 
then facilitate truly data-driven decisions. At the same time, she added, program managers 
themselves need to use the QPAS data and reports in meaningful ways (DLl interview, 
3/10/2003). 

The School Board and other high-level decision-makers have made an explicit 
connection between program accountability information and funding for programs. As indicated 
by their responses during the June 2002 and March 2003 meetings at which the QPAS updates 
were presented, the School Board indicated they plan to use the QPAS Review reports to 
facilitate informed decision-making during the annual budget process. Furthermore, analyses of 
documents, interview and meeting notes, and program manager surveys reveal that QPAS is 
impacting the presentation of the program budget and how program managers connect program 
management to program budgeting (ASl interview, 3/7/2003; AS2 interview, 4/2/2003; DLl 
interview, 3/10/2003). QPAS timelines and reporting cycles are included in the published 
program budget for the school system while at the same time budget information is incorporated 
into QPAS reports. To that end, program managers and department staff are now beginning to 
view QPAS as a vehicle through which to justify their program’s existence and to support their 
claims with evidence (AS2 interview, 4/2/2003). 

Finally, QPAS has had an impact on program improvement and functioning. Program 
managers have indicated that they use QPAS information to help them improve how their 
programs function. As the QPAS process matures, department staff expect that program 
managers will continue to expand what they are capturing in their program goals and objectives 
to represent all aspects of program functioning (DLl interview, 3/10/2003). In addition, as 
QPAS was initially developed, decisions were made about which programs should be included in 
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the first phase of QPAS, decisions that sparked discussion about how the process could be used 
to eliminate programs that are not meeting their goals and objectives. As a result of this 
discussion, program managers were asked to make explicit recommendations in the Review 
reports, based on the data presented. These recommendations are designed to facilitate informed 
decisions by high-level decision-makers about program modification, expansion, or even 
elimination. 

Conclusions 

Developing a system of shared accountability across a large educational system is a 
major undertaking. The findings from the study indicate that it takes time to change the culture 
of the system as well as the attitudes of people responsible for managing programs. QPAS has 
required a shift in thinking about what program management means, a shift requiring 
understanding and cooperation on many levels. However, once the system is in place, shared 
responsibility for accountability can be an effective way to improve programs. While some 
resistance remains, there is evidence after the first formal reporting cycle that QPAS has begun 
to increase the evaluation capacity of the entire system. Keeping QPAS viable will require an 
ongoing commitment by all involved. OPE is using the information learned in this study to 
improve the usefulness of the support and training provided to program managers, as well as how 
information about QPAS is communicated throughout the school system. Lessons learned from 
QPAS benefit not only FCPS in terms of making decisions about program improvement and 
funding, and also have implications for other educational organizations looking to develop 
shared accountability systems. 



O 

ERIC 



20 



QPAS 20 



Appendix A 
QPAS Brochure 
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Table 1 

Levels of the Quality Programs Assurance System 





Documentation 


Review 


Evaluation 


Includes: 


All instructional 
programs 


All instructional 
programs 


Selected programs 
(new, pilot, targeted) 


Responsible Party: 


Program Managers 


Program Managers 


OPE Evaluators 


Provides: 


Ongoing, current data 
upon request 


Biennial review of the 
program 


Annual interim report or 
final report 


Uses: 


Modifications for 
improved efficiency and 
effectiveness 


Modifications for 
improved efficiency and 
effectiveness 

Decisions for funding 
reallocation, expansion, 
or elimination 


Modifications for 
improved efficiency and 
effectiveness 

Decisions for funding 
reallocation, expansion, 
or elimination 

Policy decisions about 
program continuation or 
participation 
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Table 2 

Results from Initial Survey of Program Managers - November 2000 



Area 


Level of Unc 


lerstand 


ing 


Level of Comfort 


Full 

Understanding 


Some 

Understanding 


Limited 

Understanding 


No 

Understanding 


Very 

Comfortable 


Somewhat 

Comfortable 


Slightly 

Comfortable 


Not at All 
Comfortable 


My role as program manager for 
QPAS 


58% 


38% 


0% 


0% 


38% 


54% 


0% 


0% 


My responsibilities for documentation 


58% 


42% 


0% 


■ 0% 


42% 


46% 


8% 


0% 


My responsibilities for Review reports 


42% 


54% 


4% 


0% 


31% 


54% 


12% 


0% 


QPAS timelines 


62% 


38% 


0% 


0% 


38% 


58% 


0% 


0% 


QPAS Data Elements 


54% 


46% 


0% 


0% 


42% 


46% 


8% 


0% 
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Table 3 

Summary of Responses from Second Survey of Program Managers - Spring 2002 



Item 


Responses 


Percent 


At this point, at what level is your understanding of the QPAS process? 


Full Understanding 
Some Understanding 
Limited Understanding 
No Understanding 


43% 

54% 

4% 

0% 




Very Useful 


21% 


At this point, how useful has the QPAS process been to your program 


Somewhat Useful 


32% 


management? 


Slightly Useful 


21% 




Not at All Useful 


21% 




Very Useful 


54% 




Somewhat Useful 


36% 


Workshop training from OPE 


Slightly Useful 


7% 




Not at All Useful 


4% 


FoIIow-up technical assistance from OPE 


Very Useful 


57% 




Somewhat Useful 


25% 




Slightly Useful 


7% 




Not at All Useful 


0% 




Not Applicable 


11% 


Tools for QPAS (Training Handbook, Excel Workbook for maintaining 


Very Useful 


32% 


data elements) 


Somewhat Useful 


46% 




Slightly Useful 


11% 




Not at All Useful 


7% 




Not Applicable 


4% 




Very Useful 


25% 




Somewhat Useful 


29% 


Support from other departments 


Slightly Useful 


21% 




Not at All Useful 


4% 




Not Applicable 


18% 




Very Useful 


54% 




Somewhat Useful 


21% 


Support from supervisors within your department 


Slightly Useful 


14% 




Not at All Useful 


4% 




Not Applicable 


7% 



Note. Percents may not add up to 100% due to rounding. 




C 



6 



QPAS 26 



Table 4 

Summary of Responses from Third Survey of Program Managers - March 2003 



Item 


Response Choices 


Value of technical assistance beyond formal 
training: 


Very 

Useful 


Somewhat 

Useful 


Slightly 

Useful 


Not 

Useful 


Not 

Applicable 


Email exchanges 


50% 


30% 


5% 


0% 


15% 


Individual meetings 


70% 


15% 


0% 


0% 


10% 


Group meetings 


20% 


70% 


0% 


0% 


10% 
















Strongly 

Agree 


Agree 


Disagree 


Strongly 

Disagree 


Not 

Applicable 


The QPAS Review report training was useful to me 
as I completed the Review report. 


10% 


75% 


5% 


0% 


10% 


I was well prepared to complete the QPAS Review 
report for my program. 


5% 


70% 


20% 


0% 


5% 


I have a good understanding of the QPAS 
requirements. 


10% 


70% 


15% 


0% 


5% 


I received the necessary support (data, assistance, 
etc.) from OPE staff to complete the Review report. 


25% 


60% 


0% 


0% 


15% 


I received the necessary support (data, assistance, 
etc.) from within my department to complete the 
Review report. 


20% 


70% 


5% 


0% 


5% 


I received the necessary support (data, assistance, 
etc.) from other departments to complete the Review 
report. 


5% 


30% 


20% 


5% 


40% 


The QPAS Review report for my program 
accurately captures how my program worked during 
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QPAS Review reports are important to my work. 
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QPAS Review reports are important to the 
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QPAS Review reports are important to the School 
Board. 
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